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Abstract 

We present a procedure for calculating an upper limit on the number of signal events 
which incorporates the Poisson uncertainty in the background, estimated from con- 
trol regions of one or two dimensions. For small number of signal events, the upper 
limit obtained is more stringent than that extracted without including the Poisson 
uncertainty. This trend continues until the number of background events is com- 
parable with the signal. When the number of background events is comparable or 
larger than the signal, the upper limit obtained is less stringent than that extracted 
without including the Poisson uncertainty. It is therefore important to incorporate 
the Poisson uncertainty into the upper limit; otherwise the upper limit obtained 
could be too stringent. 



1 Introduction 



In the search for a rare or forbidden process, an experiment usually observes 
zero or few candidate events and sets an upper limit on the number of signal 
events. The background is often estimated from control regions of one or two 
dimensions. In the case of a small number of candidates events, it is important 
to incorporate the uncertainty in the background estimate due to Poisson 
statistics into the upper limit. 

For N ob observed events with an expected background of N bg events, the upper 
limit on the number of signal events Aq at a confidence level 5 is given by: 



-(A +JVi, a ) spN ob (Xo+N bg ) n 
P -N bg (NbsT 
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assuming that there is no uncertainty in the background estimate [1,2]. In this 
paper, we present a procedure for incorporating the statistical uncertainty in 
the background estimate using Poisson statistics. 



2 Incorporation of the statistical uncertainty 

We consider the case in which the background is estimated from control regions 
of one or two dimensions which have limited statistics. Figure 1(a) shows an 
example of a signal and two sideband regions in the distribution of a physical 
variable for the one-dimensional case. The corresponding example for the two- 
dimensional case for the signal, four sideband and four corner-band regions is 
shown in Fig. 1(b). Assuming that the background is linear in the vicinity of 
the signal region, the background in the signal region can be estimated from 
the number of events in the sideband (and corner-band if appropriate) regions. 
The number of signal events iV is given by: 

iV = N ob - N bg = N ob - aN sb + [3N cb , (2) 

where N sb (N cb ) is the number of events in the sideband (corner-band) regions. 
For the one-dimensional case, a is the ratio of the width of the signal region to 
the total width of the sideband regions and (3 is zero. In the two-dimensional 
case, a is two times the ratio of the area of the signal region to the total area 
of the sideband regions and (3 is the ratio of the area of the signal region to 
the total area of the corner-band regions. 

The extraction of the upper limit on the signal must account for the uncer- 
tainty in the background estimate due to limited statistics in the control re- 
gions. For the one-dimensional case, this is implemented by expanding Eq. (1) 
to sum over all possible background values weighted by the Poisson proba- 
bility for observing the N sb background events given an expected number of 
events X sb in the sideband regions. The upper limit on the number of signal 
events A at a confidence level 5 is given by: 

jV b 

f°° H\ At 3 p -A s 6p-(A+aA s6 ) sp N »b (A+aA st ,)" 
JO UA sb N sb \ & e 2^n=0 n \ 

Jo aA sbjj^je s »e 2^n=0 n\ 

V AU V n a k X n - k e- x (N sb +k)\ 
_ 2^n=0l^k=0 (l+q) fc fc!(n-fc)! 
_ ^ N ob a k (N sb +k)\ 

^k=0 (l +a ) k k\ 

= 1-5. 

The integral of \ sb is performed with J °° dye~ y y k 
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Fig. 1. (a) Definition of signal and two sideband regions (arrowed brackets) in the 
X distribution for the one-dimensional case. The width of the sideband regions 
is the same as the signal region, (b) Definition of signal, sideband (dashed), and 
corner-band (dotted) regions in the Y vs. Z distribution for the two-dimensional 
case. The area of a sideband or corner-band region is the same as the signal region. 



can also be used to calculate the upper limit on the number of signal events for 
the case in which the background is estimated from a Monte Carlo simulation. 
In this case, N sb is the number of background events that satisfied the event 
selection criteria in a Monte Carlo sample that is a -1 times larger than the 
data. 

For control regions of two dimensions, the formula for the upper limit is much 
more complicated. However, it can be simplified depending on the value of j3. 
In the case of f3 — 1 which corresponds a total corner-band area that is equal 
to the area of the signal region, the upper limit is given by: 



f°° d\ i. f aXsb rl\ , A «t- S r -A,f Ab C p -X cb p -(X+aX sb -X cb ) ^ N ob (X+aX sb -X cb ) n 

dX f? Xsb dX ,^^ e - X sbhdL e ~Kb e -{a\ sb -\ cb ) Y N ob (aA sb ~A eb )" 
JO u/ *sb Jo "^cfc Ngb \ c jv cfc ; c c Z^n=0 n \ 

^N ob ,pn a k X n - k e- x ^k (N sb +N cb +k+l)\ 
l^n=0 l^k=0 (l+ a ) fc (n-fe)! ^3=0 j\(k-j)\(N cb +k-j+l) 
' ^N ob gk v fc (-l)k-J(N sb +N cb +k+l)\ 

l^k=0 (i +a )k 2^j=o j ! (fc- j) ! ( jv^+fe—j+i) 

= 1-*, (4) 



where X^, is the expected number of events in the corner-band regions. Note 
that the unphysical region (X c b > aX s b) is excluded from the integral. 

For a larger corner-band area, < f3 < 1, the upper limit is given by: 
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where the unphysical region (A c & > ^A s f, = j\ s b) is excluded from the integral. 
The integral of A c6 is obtained with f* dye~y y k = k\(l - E- =0 fr^)- 

In the following sections, we consider two widely used relative dimensions 
between the signal and control regions. 

2. 1 Case I: a — \ and (5 — \ 



We first consider the one-dimensional case in which the total width of the 
sideband regions is twice that of the signal region as shown in Fig. 1(a). This 
corresponds to a — \ and Eq. (3) for the upper limit on the signal is simplified 
to: 



\ N 'b 

JO UA sb N \ 
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We now consider the two-dimensional case in which the total area of both the 
sideband and corner-band regions is four times that of the signal region as 
shown in Fig. 1(b). This corresponds to a — \ and (3 — \. From Eq. (5), the 
upper limit on the signal is given by: 
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2.2 Case II: a = (3 = 1 



We first consider the one-dimensional case in which the total width of the 
sideband regions is the same as that of the signal region (Fig. 2(a)); an exper- 
imenter may choose this kind of smaller background control regions so that 
the background is more linear in the vicinity of the signal region. Substituting 
for a = 1 in Eq. (3), the upper limit of the signal is given by: 



f^dX k ^t!L P -\s be -(x+x sb ) T N ob (\+\ ab r 

JO aA sb N i C e 2^n=0 n \ 



Jo UA sb Nsb \ e 
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We now consider the two-dimensional case in which the total area of the 
sideband regions is twice that of the signal region and the total area of corner- 
band regions is the same as that of the signal region (Fig. 2(b)). The upper 
limit is given by Eq. (4) through substitution of a — j3 — 1: 



iV ' sb N cb 

f°° rl\ i. f Xsb rl\ , A s6 3 r -^.,h X cb P -Xcb P -(A+A 36 -A e6 ) ^pN Q . 
JO aA sbJo u/\ c b jy s6 J e N cb \ 2-~in= 



JS° d\ sb jf* d\ cb ^e-^^je-^e-^ b -^ b ) E ^ i^KbT 
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Z^n=0 Z^fe=0 2 fc (n-fe)! ^i=0 j!(fc-j)!(Ar c6 +fe-j+l) 

V AU 1 V fe (-l) fc -J(JV si ,+jV ei ,+fc+l)! 
Z^fc= 2^ ^j=0 j!(fc_j)!(Ar c6 +fc-j+l) 

1 - 5. (9) 



3 Results 



We have investigated the implication of including the Poisson uncertainty in 
the estimated background in the calculation of upper limits. In Tables 1-3, we 
give the 90% and 95% confidence level upper limits on the signal with and 
without including the Poisson uncertainty for several cases of small number 
of observed events and various backgrounds. The control regions have been 
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Fig. 2. (a) Definition of signal and two sideband regions (arrowed brackets) in the 
X distribution for the one-dimensional case. The width of each sideband region 
is half that of the signal region, (b) Definition of signal, sideband (dashed), and 
corner-band (dotted) regions in the Y vs. Z distribution for the two-dimensional 
case. The area of each sideband (corner-band) region is ^ (j) that of the signal 
region. 

chosen such that ol—\ and f3 — \. In the calculation of A with Eq. (1), the 
background is set to zero if N cb > 2N sb . As expected, A is identical to A when 
the number of events in the signal region is zero, regardless of the number of 
estimated background events. However, A is smaller than A when the number 
of events in the signal region is non-zero and the estimated background is zero. 
This is not unexpected because Eqs. (6) and (7) allow for the estimated zero 
background to fluctuate up, in contrast to Eq. (1) in which the background is 
estimated to be zero with no uncertainty. This trend continues until the num- 
ber of background events is comparable with the signal. When the number 
of background events is comparable or larger than the signal, the upper limit 
obtained is less stringent than that extracted without including the Poisson 
uncertainty in the estimated background. It is therefore important to incor- 
porate the Poisson uncertainty into the upper limit; otherwise the upper limit 
obtained could be too stringent. 

For a large number of observed events, we can use the common assumption 
that the signal N in Eq. (2) is normally distributed (Gaussian) with the 
variance given by 



o 2 = N ob + ^N sb + ^N cb , (10) 

where we set a — \ and f3 — |. The upper limit on the number of signal 
events Ac is obtained by integrating from to Xq so that the integrated 
area is 5 of the area integrated to +oo. For an unphysical signal, iVo < 0, 
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Table 1 

Upper limits on the signal for a few observed events and various backgrounds in the 
one-dimensional case. The total width of the sideband regions is twice that of the 
signal region. 





90% 


C.L. 


95% 


C.L. 


(M, Esb) 


An 


A 


An 


A 


CO 


2.30 


2.30 


3.00 


3.00 


(1 (]) 


3.89 


3.61 


4.74 


4.47 




3.51 


3.42 


4.36 


4.27 




3 27 


3 27 


4.11 


4.11 




3.11 


3.16 


3.94 


3.99 


(I 2) 


3.00 


3.07 


3.82 


3.90 


(2 Q) 


5.32 


4.94 


6.30 


5.92 


(2 I) 


4.44 


4.36 


5.41 


5.33 


(2 2) 


3.88 


3.96 


4.82 


4.91 


(2 3) 


3.52 


3.68 


4.44 


4.61 


(2,4) 


3.29 


3.47 


4.18 


4.39 


(3,0) 


6.68 


6.26 


7.75 


7.34 


(3,1) 


5.71 


5.53 


6.78 


6.61 


(3,2) 


4.93 


4.96 


5.98 


6.03 


(3,3) 


4.36 


4.53 


5.40 


5.58 


(3,6) 


3.48 


3.74 


4.42 


4.73 



the integration should be renormalized by setting iVn = to obtain a more 
conservative limit [2] . Tables 4-5 show a comparison of Xq and A for the case of 
N f, = Ni,g. Due to the longer tail of the Poisson distribution, A is larger than 
Xg- The significance of the difference diminishes with larger iV {,. For example, 
in the one-dimensional case, it decreases from ~ 10% for N t> = 10 events to 
~ 5% for N ob = 50 events. For an experiment with small systematic error, 
Eq. (6) or (7) should be used to compute the upper limit instead of using the 
Gaussian approximation. 

For completeness, we also listed in Tables 6-8 the 90% and 95% confidence 
level upper limits on the signal for a — (5 — 1 with and without including the 
Poisson uncertainty in the estimated background for small number of observed 
events and various backgrounds. 

As noted in the previous section, Eq. (3) can also be used to calculate the up- 
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Table 2 

Upper limits on the signal for zero or one observed event and various backgrounds 
in the two-dimensional case. The total area of both the sideband and corner-band 
regions is four times that of the signal region. 





90% 


C.L. 


95% 


C.L. 


(N ob ,»f,»f) 


Ao 


A 


Ao 


A 


at at x 
(() ^sb "cb \ 

l u , 2 ' 4 / 


2.30 


2.30 


3.00 


3.00 


AT 

(1,0,^) 


3.89 


3.61 


4.74 


4.47 


(1,5,0) 


3.51 


3.48 


4.36 


4.33 


C1 1 -) 


3.67 


3.51 


4.53 


4.36 


(1 - -) 

>,- L ,2'2/ 


3.89 


3.53 


4.74 


4.38 


(1,2,1) 


3.89 


3.55 


4.74 


4.40 


(1,2,2) 


3.89 


3.57 


4.74 


4.43 


(1,1,0) 


3.27 


3.35 


4.11 


4.20 


(l,l,l) 


3.38 


3.40 


4.22 


4.25 


(1,1,5) 


3.51 


3.44 


4.36 


4.29 


(1,1,1) 


3.89 


3.49 


4.74 


4.33 


(1,1,2) 


3.89 


3.53 


4.74 


4.38 


(1,1,4) 


3.89 


3.57 


4.74 


4.42 


(1,2,0) 


3.00 


3.14 


3.82 


3.98 


(1 2 M 


3.05 


3.21 


3.88 


4.05 


(1,2,|) 


3.11 


3.26 


3.94 


4.11 


(1,2,1) 


3.27 


3.35 


4.11 


4.19 


(1,2,2) 


3.89 


3.44 


4.74 


4.29 


(1,2,4) 


3.89 


3.51 


4.74 


4.36 



per limit on the signal for the case in which the background is estimated from 
a Monte Carlo simulation. The impact of including the Poisson uncertainty 
in the estimated background into the upper limit can be investigated by com- 
paring the limit with that obtained without including the uncertainty. Table 9 
lists the 90% confidence level upper limits obtained with and without including 
the Poisson uncertainty for several cases of small number of observed events 
and various backgrounds. As in the previous examples, A is smaller than Ao 
for small number of observed events with comparable or smaller background. 
However, A is larger than A for larger background. Not including the Poisson 
uncertainty leads to a too stringent limit in this case. 
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Table 3 

Upper limits on the signal for two observed events and various backgrounds in the 
two-dimensional case. The total area of both the sideband and corner-band regions 
is four times that of the signal region. 





90% 


C.L. 


95% 

•J 'J / u 


C.L. 


(N, Esk Nfb\ 


An 


A 


An 


A 


(o 

l z, ) u ) 4 ) 


5.32 


4.94 


6.30 


5.92 


(2 1 0) 


4.44 


4.49 


5.41 


5.46 


(2 1 M 


4 84 


4 64 


5 82 


5 61 




5 32 


4 72 


6 30 


5 69 


(2 1 2) 


5.32 


4.80 


6.30 


5.77 


(2 1 4) 


5.32 


4.86 


6.30 


5.83 


(2 2 0) 


3.88 


4.09 


4.82 


5.05 


(2 2 -) 


4.13 


4.31 


5.08 


5.27 


(2 2 1) 


4.44 


4.45 


5.41 


5.43 


(2 2 2) 


5.32 


4.62 


6.30 


5.60 


(2 2 4) 


5.32 


4.76 


6.30 


5.74 


(2 2 6) 


5.32 


4.82 


6.30 


5.79 


(2,4,0) 


3.29 


3.55 


4.18 


4.48 


(2,4,|) 


3.39 


3.73 


4.30 


4.67 


(2,4,1) 


3.52 


3.91 


4.44 


4.86 


(2,4,2) 


3.88 


4.20 


4.82 


5.17 


(2,4,4) 


5.32 


4.52 


6.30 


5.49 


(2,4,6) 


5.32 


4.66 


6.30 


5.63 



4 Conclusion 

We have presented a procedure for calculating an upper limit on the number 
of signal events which incorporates the Poisson uncertainty in the background 
estimated from control regions of one or two dimensions. For small number of 
observed events in the signal region, the limit obtained is more stringent than 
that extracted assuming no uncertainty in the estimated background. This 
trend continues until the number of background events is comparable with 
the signal. When the number of background events is comparable or larger 
than the signal, the upper limit obtained is less stringent than that extracted 
without including the Poisson uncertainty in the estimated background. It 
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Table 4 

Upper limits on the signal for large number of observed events with large background 
in the one-dimensional case. The total width of the sideband regions is twice that 
of the signal region. 







90% C.L. 




c 


15% C.L. 






Ao 


A 


Ac 


Ao 


A 


Ac 


(10,10) 


6.63 


7.24 


6.35 


8.08 


8.78 


7.59 


(20,20) 


8.75 


9.82 


8.98 


10.60 


11.83 


10.74 


(35,35) 


11.11 


12.69 


11.88 


13.41 


15.24 


14.20 


(50,50) 


13.00 


15.00 


14.20 


15.66 


17.98 


16.97 



Table 5 

Upper limits on the signal for large number of observed events with large background 
in the two-dimensional case. The total area of both the sideband and corner-band 
regions is four times that of the signal region. 







90% C.L. 






95% C.L. 






Ao 


A 


Ac 


Ao 


A 


Ag 


(10,20,10) 


6.63 


8.61 


7.78 


8.08 


10.31 


9.30 


(15,25,10) 


7.78 


9.86 


8.98 


9.45 


11.82 


10.74 


(20,30,10) 


8.75 


10.93 


10.04 


10.60 


13.10 


12.00 



is therefore important to incorporate the Poisson uncertainty into the upper 
limit; otherwise the upper limit obtained could be too stringent. 



5 Acknowledgment 

This work was supported in part by the U.S. Department of Energy. K.K.G. 
thanks the OJI program of DOE for support. 



References 

[1] G. Zech, Nucl. Instr. and Meth. A277 (1989) 608. 

[2] R.M. Barnett et al, Review of Particle Physics, Phys. Rev. D54, (1996) 1. 



10 



Table 6 

Upper limits on the signal for a few observed events and various backgrounds in the 
one-dimensional case. The total width of the sideband regions is the same as that 
of the signal region. 

90% C.L. 95% C.L. 



(N ob ,N sb ) 


Ao 


A 


Ao 


A 


(0,N sb ) 


2.30 


2.30 


3.00 


3.00 


(!>0) 


3.89 


3.51 


4.74 


4.36 


(M) 


3.27 


3.27 


4.11 


4.11 


(i, 2 ) 


3.00 


3.11 


3.82 


3.94 


(M) 


2.84 


3.00 


3.64 


3.82 


/ 1 A \ 

(M) 


2.74: 


2.91 


3.53 


3.72 


(!>6) 


2.62 


2.78 


3.39 


3.58 


( 2 ,0) 


5.32 


A TV 

4.75 


f* on 

6.30 


5.72 


( 2 ,1) 


4.44 


4.32 


5.41 


5.29 


( 2 , 2 ) 


3.88 


4.01 


4.82 


4.97 


(2,3) 


3.52 


3.77 


4.44 


4.72 


( 2 ,4) 


3.29 


3.59 


4.18 


4.52 


( 2 ,6) 


3.01 


3.33 


3.86 


4.23 


(3,0) 


6.68 


5.99 


7.75 


7.08 


(3,1) 


5.71 


5.43 


6.78 


6.52 


(3, 2 ) 


4.93 


4.99 


5.98 


6.06 


(3,3) 


4.36 


4.63 


5.40 


5.69 


(3,4) 


3.97 


4.35 


4.97 


5.39 


(3,6) 


3.48 


3.93 


4.42 


4.94 
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Table 7 

Upper limits on the signal for zero or one observed event and various backgrounds 
in the two-dimensional case. The total area of the sideband regions is twice that of 
the signal region while the total area of the corner-band regions is the same as that 
of the signal region. 





90% C.L. 


95% C.L. 


(N ob ,N sb ,N cb ) 


Ao 


A 


Ao 


A 


(0,N sb ,N cb ) 


2.30 


2.30 


3.00 


3.00 


(l,0,N cb ) 


3.89 


3.51 


4.74 


4.36 


(1,1,0) 


3.27 


3.38 


4.11 


4.22 


(1,1,1) 


3.89 


3.42 


4.74 


4.27 


(1,1,2) 


3.89 


3.44 


4.74 


4.29 


(1,1,3) 


3.89 


3.45 


4.74 


4.30 


(1,1,4) 


3.89 


3.46 


4.74 


4.31 


(1,1,6) 


3.89 


3.47 


4.74 


4.32 


(1,2,0) 


3.00 


3.27 


3.82 


4.11 


(1,2,1) 


3.27 


3.34 


4.11 


4.18 


(1,2,2) 


3.89 


3.38 


4.74 


4.22 


(1,2,3) 


3.89 


3.40 


4.74 


4.25 


(1,2,4) 


3.89 


3.42 


4.74 


4.27 


(1,2,6) 


3.89 


3.44 


4.74 


4.29 


(1,3,0) 


2.84 


3.18 


3.64 


4.02 


(1,3,1) 


2.99 


3.27 


3.82 


4.11 


(1,3,2) 


3.27 


3.32 


4.11 


4.17 


(1,3,3) 


3.89 


3.36 


4.74 


4.20 


(1,3,4) 


3.89 


3.38 


4.74 


4.22 


(1,3,6) 


3.89 


3.41 


4.74 


4.25 
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Table 8 

Upper limits on the signal for two observed events and various backgrounds in the 
two-dimensional case. The total area of the sideband regions is twice that of the 
signal region while the total area of the corner-band regions is the same as that of 
the signal region. 

90% C.L. 95% C.L. 



(No^N^Nj,) 


Ao 


A 


Ao 


A 


(2,0,N cb ) 


5.32 


4.75 


6.30 


5.72 


(2,1,0) 


4.44 


4.50 


5.41 


5.47 


(2,1,1) 


5.32 


4.57 


6.30 


5.55 


(2,1,2) 


5.32 


4.61 


6.30 


5.59 


(2,1,3) 


5.32 


4.64 


6.30 


5.61 


(2,1,4) 


5.32 


4.65 


6.30 


5.63 


(2,1,6) 


5.32 


4.68 


6.30 


5.65 


(2,2,0) 


3.88 


4.29 


4.82 


5.26 


(2,2,1) 


4.44 


4.41 


5.41 


5.39 


(2,2,2) 


5.32 


4.49 


6.30 


5.46 


(2,2,3) 


5.32 


4.53 


6.30 


5.51 


(2,2,4) 


5.32 


4.56 


6.30 


5.54 


(2,2,6) 


5.32 


4.61 


6.30 


5.58 


(2,3,0) 


3.52 


4.10 


4.44 


5.07 


(2,3,1) 


3.88 


4.27 


4.82 


5.24 


(2,3,2) 


4.44 


4.37 


5.41 


5.34 


(2,3,3) 


5.32 


4.43 


6.30 


5.41 


(2,3,4) 


5.32 


4.48 


6.30 


5.45 


(2,3,6) 


5.32 


4.54 


6.30 


5.52 


(2,4,0) 


3.29 


3.94 


4.18 


4.90 


(2,4,1) 


3.52 


4.14 


4.44 


5.10 


(2,4,2) 


3.88 


4.26 


4.82 


5.23 


(2,4,3) 


4.44 


4.34 


5.41 


5.31 


(2,4,4) 


5.32 


4.40 


6.30 


5.37 


(2,4,6) 


5.32 


4.48 


6.30 


5.45 
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Table 9 

90% confidence level upper limits on the signal for small number of observed events 
and various backgrounds estimated from a Monte Carlo Simulation. 



3 6 9 
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O.Z 1 


Q /10 
0.4Z 


q /in 
o.4U 


f1 


z.yo 


o.UU 


O.Z1 


Q OO 
o.ZZ 


Q Q7 
O.O 1 


^ft 
o.oO 


(d,°) 


O fifi 
Z.OO 


o 

z.yo 


Q 1 ft 
O.lO 


Q 1 7 
O.l / 


Q QO 
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Q Q1 

o.ol 




9 fi/l 
Z.o4 


z.y i 


O 1 1 
O.ll 


o iq 
O.lO 


07 
O.Z f 


07 
O.Z f 


(2,0) 


5.32 


5.04 


5.32 


5.17 


5.32 


5.22 


(2,1) 


5.00 


4.79 


5.16 


5.02 


5.21 


5.11 


(2,2) 


4.70 


4.57 


5.00 


4.88 


5.10 


5.01 


(2,3) 


4.44 


4.38 


4.84 


4.75 


5.00 


4.92 


(2,4) 


4.22 


4.21 


4.70 


4.63 


4.89 


4.82 


(2,5) 


4.04 


4.07 


4.57 


4.51 


4.79 


4.73 


(2,6) 


3.88 


3.94 


4.44 


4.41 


4.70 


4.65 


(2,7) 


3.74 


3.82 


4.33 


4.31 


4.61 


4.57 


(2,8) 


3.62 


3.72 


4.22 


4.22 


4.52 


4.49 


(2,9) 


3.52 


3.63 


4.13 


4.13 


4.44 


4.42 
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